A Latent Trait Method for Determining Incon- Sistences in the Use of the Angoff and Nedel Sky Techniques of Standard Setting

It has often been argued that all techniques of standard setting are arbitrary and likely to yield different results for different techniques or persons. This paper deals with a related but hitherto ignored aspect of standard setting, namely the possibility that Angoff or Nedelsky judges misspecify the probabilities of the borderline student's success on the items because they do not use the psychometric properties of the items consistently. A latent trait method is proposed to estimate such misspecifications and an index of consistency is defined which can be used for deciding whether standards are set consistently enough for use in practice. Also, results from an empirical study are given which illustrate the use of the method in a typical educational situation. Thy results indicate that serious errors of specification can be expected and that, on the whole, these will be considerably larger for the Nedelsky than for the Angoff technique.